Performance tuning for vec search

[arkohut]

Hi, I’ve been using sqlite-vec to build my personal screen retrieval project, Pensieve, but I’ve encountered some performance-related issues recently.

Currently, I’m running this project on a M1 Max MacBook, with approximately 250,000 records in the database. When performing vector searches, it has started to become very slow. Here’s a rough summary of the situation:

I’m not sure if this is a normal performance level? Below is my table structure and the corresponding SQL:

  CREATE VIRTUAL TABLE entities_vec_v2 USING vec0(
      embedding float[1024] distance_metric=cosine,
      file_type_group text,
      created_at_timestamp integer,
      file_created_at_timestamp integer,
      file_created_at_date text partition key,
      app_name text,
      library_id integer
  )

  SELECT rowid
    FROM entities_vec_v2
    WHERE embedding MATCH ?
      AND file_type_group = 'image'
      AND K = 48
      AND library_id IN ?
    ORDER BY distance ASC

[brankoradovanovic-mcom]

No, this is not normal performance. I've tried sqlite-vec with 400,000 rows (embedding size=768), and a query like this runs in less than a second on a warmed-up database (i.e. cached, without I/O).